Papers with updating model parameters
Divide, Reweight, and Conquer: A Logit Arithmetic Approach for In-Context Learning (2026.eacl-long)
Copied to clipboard
| Challenge: | In-Context Learning (ICL) is a key feature for Large Language Models (LLMs) but it faces challenges when dealing with increasing numbers of examples due to performance degradation and quadratic computational costs. |
| Approach: | They propose a Logit Arithmetic Reweighting Approach that uses logit-based ensembling of multiple demonstrations to enhance ICL. |
| Outcome: | The proposed framework outperforms baseline methods in accuracy and memory efficiency. |
Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models (2024.findings-acl)
Copied to clipboard
Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, Jun Zhao
| Challenge: | Existing methods to expand internal memory boundaries of language models by providing external context can often conflict, leading to knowledge conflicts. |
| Approach: | They propose a method that prunes conflicting attention heads without updating model parameters. |
| Outcome: | The proposed method can flexibly control eight LMs to use internal memory or external context without updating model parameters. |
Activation Steering for Chain-of-Thought Compression (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models produce intermediate explanations, commonly referred to as chains of thought (CoTs), but the generated rationales are typically verbose, consuming many additional tokens, and thus degrading throughput and increasing inference energy consumption. |
| Approach: | They propose to generate concise reasoning traces by directly adjusting internal representations via activation steering. |
| Outcome: | The proposed method reduces generated token length by 69.4% across five reasoning benchmarks while maintaining accuracy. |
Decoding-Unlearning: Fact Forgetting via Entropy-Guided Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for large-scale modeling memorize sensitive information . however, they are limited in real-world scenarios and require updating parameters . |
| Approach: | They propose a training-free, plug-and-play inference-time unlearning strategy that uses a probe to detect queries involving forgettable concepts and applies entropy-guided decoding to suppress target knowledge. |
| Outcome: | Experiments on MUSE, RWKU, and WMDP datasets show that SEGUE outperforms existing methods. |
CAP: Controllable Alignment Prompting for Unlearning in LLMs (2026.acl-long)
Copied to clipboard
Zhaokun Wang, Jinyu Guo, Jingwen Pu, Hongli Pu, Meng Yang, Xunlei Chen, Jie Ou, Wenyi Li, Guangchun Luo, Wenhong Tian
| Challenge: | Existing methods for modifying parameters are unsystematic and rely on empirical experience. |
| Approach: | They propose a controllable alignment prompting for unlearning framework that decouples unlearning into a learnable prompt optimization process via reinforcement learning. |
| Outcome: | The proposed framework achieves precise, controllable unlearning without updating model parameters. |